Multimodal Interaction: Methods and Applications for Joint Cooperation between Humans and Cognitive Systems

In this paper, we present an overview on recent advances in our work on multimodal human-machine communication between humans and technical cognitive systems in order to enable the solution of complex problems that require the joint cooperation of humans and intelligent systems to accomplish challenging tasks that can be only successfully handled if both interact in a cooperative manner. The technical system has cognitive capabilities that support an intelligent interaction with the human operator. An example for such cooperation is the joint assembly of a heavy part by a human assisted by a cognitive robot during the manufacturing process of a complex product. As will be explained, such challenges in joint cooperation occur often in intelligent manufacturing environments which provide a very rich application scenario for the successful combination of advanced multimodal human-machine communication techniques together with cognitive components for improved, cooperative interaction between technical systems and human operators.